Skip to content

refactor(ci): consolidate yq and bicep installs into make targets#12248

Merged
DariuszPorowski merged 6 commits into
mainfrom
dp/underlying-firefly-aquamarine
Jun 25, 2026
Merged

refactor(ci): consolidate yq and bicep installs into make targets#12248
DariuszPorowski merged 6 commits into
mainfrom
dp/underlying-firefly-aquamarine

Conversation

@DariuszPorowski

Copy link
Copy Markdown
Member

Description

Problem

Tool installation (yq and the Bicep CLI) was duplicated across many GitHub workflows using inline, hand-written download steps. This created several recurring problems:

  • Version drift — yq versions were scattered across workflows and the Bicep CLI was installed from an unpinned latest, so different jobs could run different tool versions.
  • No integrity verification — inline installs downloaded binaries without checksum validation, leaving a supply-chain gap.
  • Duplicated, hard-to-maintain logic — the same install snippets were copy-pasted into multiple workflows, so any change (version bump, URL change, platform fix) had to be made in many places.
  • No single source of truth — there was no one place to see or update the tool versions the project depends on.

Value proposition

This change centralizes tool installation behind Make targets backed by a single source of truth, making CI reproducible, verifiable, and easy to maintain:

  • build/tools.mk as the single source of truth for tool versions and per-platform SHA-256 checksums (yq v4.53.3, Bicep v0.42.1).
  • Reusable, hardened installer scriptsbuild/scripts/install-yq.sh and build/scripts/install-bicep.sh install into a user-owned bin dir (no sudo), support Linux and macOS on amd64/arm64, verify checksums, honor GITHUB_TOKEN for GitHub API calls, and fall back to the latest release when no version is pinned.
  • make install-yq / make install-bicep targets — every workflow now calls these instead of bespoke inline steps, so versions are pinned consistently and updated in one place.
  • De-duplicated Bicep toolingbuild/install-bicep.sh is reduced to its sole remaining responsibility (generating bicepconfig.json); all download/install logic now lives in the single build/scripts/install-bicep.sh. build/build.mk calls the installer for cross-arch container staging.
  • Stronger fork safetyfunctional-test-cloud.yaml gains an explicit authorize gate so external/fork contributions run only after the trust check and approval gate pass.
  • CI cleanup — expands the __changes.yml ignore-list, normalizes the unit-tests concurrency group, relocates the triage-bot label config to .github/configs/label-actions.yaml, and removes the obsolete root CODEOWNERS.

Changed files

  • Added: build/tools.mk, build/scripts/install-yq.sh, build/scripts/install-bicep.sh
  • Modified (build): Makefile, build/build.mk, build/generate.mk, build/resource-types.mk, build/install-bicep.sh (stripped to bicepconfig.json generation)
  • Modified (workflows): copilot-setup-steps.yml, functional-test-cloud.yaml, functional-test-noncloud.yaml, validate-bicep.yaml, contrib-update-resource-types.yaml, verify-resource-types.yaml, lint.yaml, publish-docs.yaml, release.yaml, triage-bot.yaml, unit-tests.yaml, __changes.yml
  • Renamed: .github/triage-bot/triage-bot-config.yaml.github/configs/label-actions.yaml
  • Deleted: root CODEOWNERS

Type of change

  • This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

N/A — minor CI/build tooling refactor (issue link optional).

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

  • An overview of proposed schema changes is included in a linked GitHub issue.
    • Yes
    • Not applicable
  • A design document is added or updated under eng/design-notes/ in this repository, if new APIs are being introduced.
    • Yes
    • Not applicable
  • The design document has been reviewed and approved by Radius maintainers/approvers.
    • Yes
    • Not applicable
  • A PR for resource-types-contrib is created, if resource types or recipes are affected by the changes in this PR.
    • Yes
    • Not applicable
  • A PR for dashboard is created, if the Radius Dashboard is affected by the changes in this PR.
    • Yes
    • Not applicable
  • A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
    • Yes
    • Not applicable

Copilot AI review requested due to automatic review settings June 25, 2026 06:45
@DariuszPorowski DariuszPorowski requested review from a team as code owners June 25, 2026 06:45
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

  • .github/workflows/release.yaml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes installation of CI tooling (notably yq and the Bicep CLI) behind new Make targets and hardened installer scripts, replacing duplicated inline download logic across multiple GitHub workflows and build targets.

Changes:

  • Add build/tools.mk and new installer scripts to pin tool versions and (where possible) verify SHA-256 checksums.
  • Update build Make targets and multiple workflows to use make install-yq / make install-bicep instead of bespoke inline install steps.
  • CI cleanup: adjust workflow concurrency/skip lists, move triage-bot label config, and remove the obsolete root CODEOWNERS.

Review notes (blocking):

  • build/build.mk currently passes "$$(BICEP_VERSION)"-style values, which becomes shell command substitution ($(...)) and will fail at runtime.
  • .github/workflows/release.yaml introduces ::set-output, which is deprecated in GitHub Actions; it should use $GITHUB_OUTPUT.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
Makefile Includes build/tools.mk to expose shared tool install targets.
CODEOWNERS Removes obsolete root-level pointer to .github/CODEOWNERS.
build/tools.mk Defines pinned yq / bicep versions + checksums and provides install-* Make targets.
build/scripts/install-yq.sh New cross-platform installer for yq with checksum verification.
build/scripts/install-bicep.sh New cross-platform installer for Bicep CLI with optional checksum verification and cross-arch staging support.
build/resource-types.mk Updates yq prerequisite message to use make install-yq.
build/install-bicep.sh Narrows script responsibility to only generating bicepconfig.json.
build/generate.mk Removes old YQ_VERSION pin and updates install guidance to make install-yq.
build/build.mk Updates bicep container staging to use the new Bicep installer script.
.github/workflows/verify-resource-types.yaml Replaces inline yq install with make install-yq.
.github/workflows/validate-bicep.yaml Replaces inline bicep install with make install-bicep (installing into ~/.rad/bin).
.github/workflows/unit-tests.yaml Normalizes concurrency group expression.
.github/workflows/triage-bot.yaml Points label-actions config to new .github/configs/label-actions.yaml location.
.github/workflows/release.yaml Switches from action-based yq usage to installing yq and invoking it directly (but uses deprecated ::set-output).
.github/workflows/publish-docs.yaml Replaces inline yq install with make install-yq.
.github/workflows/lint.yaml Replaces inline yq install with make install-yq.
.github/workflows/functional-test-noncloud.yaml Replaces inline yq/bicep installs with Make targets.
.github/workflows/functional-test-cloud.yaml Adds centralized authorize gate and replaces inline tool installs with Make targets.
.github/workflows/copilot-setup-steps.yml Replaces inline yq/bicep installs with Make targets.
.github/workflows/contrib-update-resource-types.yaml Replaces inline yq install with make install-yq.
.github/workflows/__changes.yml Expands ignore list and adds documentation clarifying skip behavior.
.github/configs/label-actions.yaml New location/name for triage-bot label-actions config (plus small YAML cleanup).

Comment thread build/build.mk Outdated
Comment thread .github/workflows/release.yaml Outdated
Comment thread .github/workflows/release.yaml Outdated
- Removed hardcoded yq installation steps from multiple workflows and replaced them with a single `make install-yq` command.
- Updated Bicep CLI installation to use `make install-bicep` in relevant workflows.
- Removed deprecated environment variables related to yq and Bicep versions from workflows.
- Added new scripts for installing yq and Bicep CLI, allowing for easier version management and installation.
- Updated Makefile to include new tools.mk for managing tool versions and checksums.
- Adjusted concurrency group format in unit-tests workflow for improved consistency.
- Changed CODEOWNERS file path in triage-bot workflow.
- Deleted obsolete CODEOWNERS file.

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…ully

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…e workflow

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
@DariuszPorowski DariuszPorowski force-pushed the dp/underlying-firefly-aquamarine branch from a3810bd to d71c49e Compare June 25, 2026 06:54
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Unit Tests

    2 files  ±0    450 suites  ±0   7m 24s ⏱️ -13s
5 591 tests ±0  5 589 ✅ ±0  2 💤 ±0  0 ❌ ±0 
6 788 runs  ±0  6 786 ✅ ±0  2 💤 ±0  0 ❌ ±0 

Results for commit 21c73ad. ± Comparison against base commit 2626297.

♻️ This comment has been updated with latest results.

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.87%. Comparing base (2626297) to head (21c73ad).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #12248      +/-   ##
==========================================
- Coverage   52.88%   52.87%   -0.02%     
==========================================
  Files         751      751              
  Lines       48353    48353              
==========================================
- Hits        25573    25566       -7     
- Misses      20383    20387       +4     
- Partials     2397     2400       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@brooke-hamilton brooke-hamilton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

One non-blocking suggestion:

Rename the bicepconfig generator. After this change, build/install-bicep.sh no longer installs the Bicep CLI — it only generates bicepconfig.json — while the new build/scripts/install-bicep.sh does the actual CLI install. Two files sharing the basename install-bicep.sh with unrelated jobs is easy to confuse. Consider renaming build/install-bicep.sh to something like build/gen-bicepconfig.sh and updating its caller in build/build.mk.

…date install process

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
…ation

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
@radius-functional-tests

radius-functional-tests Bot commented Jun 25, 2026

Copy link
Copy Markdown

Radius functional test overview

🔍 Go to test action run

Click here to see the test run details
Name Value
Repository radius-project/radius
Commit ref 21c73ad
Unique ID func4b494c8dd3
Image tag pr-func4b494c8dd3
  • KinD: v0.29.0
  • Dapr: 1.14.4
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func4b494c8dd3
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func4b494c8dd3
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func4b494c8dd3
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func4b494c8dd3
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func4b494c8dd3
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@DariuszPorowski DariuszPorowski added this pull request to the merge queue Jun 25, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 25, 2026
@DariuszPorowski DariuszPorowski added this pull request to the merge queue Jun 25, 2026
@DariuszPorowski DariuszPorowski removed this pull request from the merge queue due to a manual request Jun 25, 2026
@DariuszPorowski DariuszPorowski merged commit b968326 into main Jun 25, 2026
75 checks passed
@DariuszPorowski DariuszPorowski deleted the dp/underlying-firefly-aquamarine branch June 25, 2026 16:32
DariuszPorowski added a commit that referenced this pull request Jun 25, 2026
…s into make targets (#12251)

## Description

Follow-up to #12248, which consolidated the `yq` and `bicep` installs
into pinned, checksum-verified `make install-<tool>` targets. This PR
applies the same pattern to the remaining CI tools — **kind**,
**kubectl**, **dapr**, **helm**, and **oras** — replacing inline
`curl`/`wget | bash` installs and the third-party `azure/setup-helm` and
`oras-project/setup-oras` actions.

Each tool now has:

- a generic, no-sudo `build/scripts/install-<tool>.sh` that detects
OS/arch, verifies the download against a SHA-256 checksum, installs into
a user-owned dir, and adds it to `$GITHUB_PATH` for later steps; and
- a `build/tools.mk` block that pins the version + per-platform
checksums and exposes a `make install-<tool>` target.

Versions pinned in this PR: `kind` v0.32.0, `kubectl` v1.36.2, `dapr`
v1.18.0, `helm` v4.2.2, `oras` v1.3.2.

### Why

- **Single source of truth** for tool versions in `build/tools.mk`,
instead of versions scattered across workflow `env:` blocks and step
inputs.
- **Supply-chain hardening**: every binary is checksum-verified; the
unpinned `dapr` `install.sh | bash` pipe and the `setup-helm` /
`setup-oras` third-party actions are removed.
- **Consistent local + CI behavior**: the same `make install-<tool>`
runs on a contributor's machine and on the runner.

### Files

- New: `build/scripts/install-{kind,kubectl,dapr,helm,oras}.sh`
- `build/tools.mk`: new KIND / KUBECTL / DAPR / HELM / ORAS version +
checksum blocks and `install-*` targets
- Workflows/action switched to `make install-<tool>` (dropping the
now-unused version env vars): `build.yaml`, `lint.yaml`,
`unit-tests.yaml`, `copilot-setup-steps.yml`,
`functional-test-cloud.yaml`, `functional-test-noncloud.yaml`,
`validate-installers.yaml`, `actions/create-kind-cluster/action.yaml`

## Type of change

- This pull request is a minor refactor, code cleanup, test improvement,
or other maintenance task and doesn't change the functionality of Radius
(issue link optional).

Follow-up to #12248.

## Contributor checklist
Please verify that the PR meets the following requirements, where
applicable:

- An overview of proposed schema changes is included in a linked GitHub
issue.
    - [ ] Yes
    - [x] Not applicable
- A design document is added or updated under `eng/design-notes/` in
this repository, if new APIs are being introduced.
    - [ ] Yes
    - [x] Not applicable
- The design document has been reviewed and approved by Radius
maintainers/approvers.
    - [ ] Yes
    - [x] Not applicable
- A PR for
[resource-types-contrib](https://github.com/radius-project/resource-types-contrib/)
is created, if resource types or recipes are affected by the changes in
this PR.
    - [ ] Yes
    - [x] Not applicable
- A PR for [dashboard](https://github.com/radius-project/dashboard/) is
created, if the Radius Dashboard is affected by the changes in this PR.
    - [ ] Yes
    - [x] Not applicable
- A PR for the [documentation
repository](https://github.com/radius-project/docs) is created, if the
changes in this PR affect the documentation or any user facing updates
are made.
    - [ ] Yes
    - [x] Not applicable

---------

Signed-off-by: Dariusz Porowski <3431813+DariuszPorowski@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants